Neural network frequency control and compensation of control voltage linearity

ABSTRACT

Systems and methods of using an artificial neural network processing module to compensate a control voltage and create a linear output response for an electronic oscillator to produce a target frequency. The artificial neural network processing module includes one or more neurons which receive one or more inputs corresponding to the control voltage. The artificial neural network processing module is configured to provide a correction based at least in part on the control voltage and pre-calculated DAC values. The pre-calculated DAC values are determined in part by predetermined or predefined pull ranges and linear control voltage transfer functions. The artificial neural network processing module can preferably achieve a control voltage tuning linearity better than 0.5% linearity over an entire tuning range of +−75 ppm.

RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.13/570,563, filed Aug. 9, 2012, which claims priority from U.S.Provisional Patent Application Nos. 61/522,599, filed Aug. 11, 2011, and61/650,408, filed May 22, 2012. Further, this application is acontinuation-in-part of U.S. application Ser. No. 14/336,951, filed Jul.21, 2014, which claims priority from U.S. Provisional Patent ApplicationNo. 61/858,070, filed Jul. 24, 2013, and is a continuation-in-part ofU.S. application Ser. No. 13/570,563, filed Aug. 9, 2012, which claimspriority from U.S. Provisional Patent Application Nos. 61/522,599, filedAug. 11, 2011, and 61/650,408, filed May 22, 2012. Moreover, thisapplication claims priority from U.S. Provisional Patent Application No.61/915,992, filed Dec. 13, 2013. Each of the applications listed aboveare expressly incorporated by reference herein in its entirety and madea part of the present specification.

BACKGROUND

1. Field

This disclosure relates generally to frequency control, and moreparticularly to control of electronic oscillator output.

2. Description of Related Art

Neural networks can be found abundantly in nature. The brains of allanimals including humans comprise vast neural networks. Biologicalneural networks comprise biological neurons that are chemicallyconnected or functionally related in the peripheral nervous system orthe central nervous system of certain living creatures. A single neuronmay be connected to many other neurons and the total number of neuronsand connections in a network, called synapses, may be extensive. Apartfrom electrical signaling, there are other forms of signaling that arisefrom neurotransmitter diffusion, which have an effect on electricalsignaling. As such, neural networks are extremely complex. As we knowfrom the human condition, neural networks are extremely efficient atlearning and adapting to changing environments, and more proficient thancomputers at certain difficult tasks such as image recognition. As anexample of the utilization of neural networks in nature, bats fly athigh speed and use their neural network to process their ultrasonicemissions to map their environment, calculate pursuit algorithms, andfind and capture prey without injuring themselves, among other things.

Artificial Neural Networks (ANN's), also referred to herein as neuralnetworks (NN's), are mankind's attempt at replicating nature's elegantsolution. An ANN is a mathematical model or computational model that ismodeled after, or inspired by, the structure and functionality ofbiological neural networks. To date, currently implemented ANN's havenot provided the efficiency or computational power of the human brain.

SUMMARY

For purposes of summarizing the disclosure, certain aspects, advantagesand novel features have been described herein. It is to be understoodthat not necessarily all such advantages can be achieved in accordancewith any particular embodiment disclosed herein. Thus, the featuresdisclosed herein can be embodied or carried out in a manner thatachieves or optimizes one advantage or group of advantages as taughtherein without necessarily achieving other advantages as can be taughtor suggested herein.

Certain embodiments disclosed herein provide a process of controllingfrequency output of an electronic oscillator to compensate for effectsof a parameter experienced by the oscillator. The process may includereceiving test data from an electronic oscillator, the electronicoscillator being subject to a parameter, wherein the test datacorresponds to outputs generated by the electronic oscillator whilebeing subject to the parameter over a range of parameter values.Furthermore, the process may include calculating a plurality of weightsbased at least in part on the test data and receiving from theelectronic oscillator one or more input signals corresponding to acurrent value of the parameter. The one or more input signals may beprovided to a neural network processing module including one or moreprocessors. In addition, the plurality of weights may be provided to theneural network processing module.

The process may include using the neural network processing module tocalculate a neural network output signal based at least in part on theone or more input signals and the plurality of weights and determining acorrection voltage based at least in part on the neural network outputsignal. The correction voltage may, for example, be provided to theelectronic oscillator, thereby at least partially controlling thefrequency of the electronic oscillator. In certain embodiments,calculating a plurality of weights is performed prior to receiving oneor more input signals.

The process may include generating an intelligent seed value, whereincalculating the plurality of weights is based at least in part on theseed value. Calculating the neural network output signal may includeproviding the input signals and the weights to a plurality of artificialneuron modules of the neural network processing module, wherein at leastone of the weights is provided to each of the plurality of artificialneuron modules. The process may further include providing a neuronoutput from each of the plurality of neuron modules to a linear summermodule, wherein the neural network output signal is based at least inpart on an output of the linear summer module. A bias input may beprovided to the linear summer for shifting the output of the linearsummer up or down. In certain embodiments, each of the plurality ofartificial neuron modules utilizes at least one of the weights and eachof the input signals as variables of a non-linear activation function,wherein each of the plurality of artificial neuron modules provides anoutput that is based at least in part on a solution of the non-linearactivation function. The activation function may advantageously becontinuous and differentiable, such as a sigmoid function.

Certain embodiments provide a system for controlling frequency output ofan electronic oscillator. The system may include a first input signalcorresponding to a first parameter of an electronic oscillator and aneural network processing module operatively coupled to the first inputsignal, the neural network processing module including a plurality ofartificial neuron modules, wherein each of the plurality of artificialneuron modules receives the first input signal and one of one or morepre-calculated weights, and provides a first output signal based atleast in part on the first input signal and the one of the one or morepre-calculated weights. The pre-calculated weights may be derived atleast in part from test data that corresponds to outputs generated bythe electronic oscillator while experiencing a range of values of thefirst parameter. Furthermore, the neural network processing module maybe configured to provide a second output signal to the electronicoscillator to thereby at least partially control frequency output of theelectronic oscillator.

The second output may not be based on a real-time oscillating signaloutput of the electronic oscillator. The parameter may be related to atemperature of at least a portion of the electronic oscillator. Incertain embodiments, the system further includes a second input signalcorresponding to a second parameter, wherein each of the plurality ofartificial neuron modules receives the second input signal, the firstoutput signal being based at least in part on the second input signal.The system may further include a third input signal corresponding to athird parameter, wherein each of the artificial neuron modules receivesthe third input signal, the first output signal being based at least inpart on the third input signal. The neural network processing module maybe configured to generate a second output signal and provide the secondoutput signal to a voltage-conversion module configured to convert thesecond output signal into a correction voltage, and provide thecorrection voltage to the electronic oscillator.

Certain embodiments disclosed herein provide a system for providing acorrection signal to at least partially compensate for effects of aparameter of an electronic device. The system may include a first neuralnetwork processing module including one or more processors, theprocessing module configured to receive a first input signalcorresponding to a parameter. The first neural network processing modulemay include a plurality of artificial neuron modules, wherein each ofthe plurality of artificial neuron modules receives one of one or morepre-calculated weights, and wherein the first neural network processingmodule is configured to provide a correction signal based at least inpart on the first input signal and the one or more pre-calculatedweights.

The neural network processing module may be configured to provide thecorrection signal to an electronic oscillator, the parameter beingassociated with the electronic oscillator. The first input signal maycorrespond to a temperature of at least a portion of the electronicoscillator. The neural network processing module may be furtherconfigured to receive a second input signal, the second input signalrelating to an age of the electronic oscillator.

In certain embodiments, the system includes a memory module, ananalog-to-digital converter, a digital-to-analog converter, and alow-pass filter. The first input signal may be provided by a motionsensor, such as, for example, an accelerometer. Alternatively, the firstinput signal may be provided by a pressure sensor. The first parametermay be hysteresis effects experienced by an electronic oscillator. Thesystem may further include a second neural network processing moduleconfigured to receive the first input signal and provide a second outputsignal. The second output signal may be configured to at least partiallycontrol a heater device associated with the electronic oscillator.

Some embodiments disclosed herein provide a system for controllingfrequency output of an electronic oscillator. The system may include anelectronic signal input indicative of a parameter of an electronicoscillator and a neural network processing module operatively coupled tothe electronic signal input. Furthermore, the neural network processingmodule may include a plurality of artificial neuron modules, whereineach of the plurality of artificial neuron modules receives at least oneinput signal, multiplies the at least one input signal by apre-calculated weight, and provides a first output signal based, atleast in part, on the electronic signal input indicative of a parameterof the electronic oscillator. The neural network processing module mayproduce a second output signal, and provide the second output signal toa voltage conversion module configured to convert the second outputsignal into a correction voltage, and provide the correction voltage tothe electronic oscillator.

In some embodiments, a process of controlling the frequency of anelectronic oscillator may include: receiving data indicative of anoutput of the electronic oscillator over a range of temperatures;calculating a plurality of weights based, at least in part, on the data;receiving one or more input signals indicative of a parameter of theelectronic oscillator; providing the one or more input signals to aneural network processing module; providing the weights to the neuralnetwork processing module; calculating a neural network output signalbased, at least in part, on the one or more first signals and theplurality of weights; and using the neural network output signal toprovide a correction voltage to the electronic oscillator to control thefrequency of the electronic oscillator.

Some embodiments provide an electronic oscillator assembly including anelectronic oscillator having a first output including an AC waveform anda second output providing a signal indicative of a parameter of theoscillator, wherein the oscillator is configured to receive a correctionvoltage signal. The system may further include a neural networkprocessor configured to receive the second output and produce a thirdoutput based, at least in part, on the second output and a plurality ofweights. The system may further include a voltage converter operativelyconnected to the oscillator and the neural network processor andconfigured to convert the third output into the correction voltagesignal.

Some embodiments provide a system for adjusting a frequency of anelectronic oscillator, wherein the system provides a neural networkprocessor operative to receive an electronic temperature signalindicative of a temperature of the electronic oscillator. The neuralnetwork may be further operative to process the temperature signal byproviding the temperature signal to a plurality of artificial neuronmodules, wherein each of the artificial neuron modules produces anoutput signal by operating on the temperature signal using asubstantially sigmoid-shaped activation function, wherein each of thesigmoid-shaped activation functions used by the plurality of artificialneuron modules is pre-calculated. The neural network may be furtheroperative to provide the output signals produced by each of theplurality of artificial neuron modules to a linear summation moduleconfigured to produce an output based, at least in part, on the outputsignals produced by the plurality of artificial neuron modules. In someembodiments, the system includes a voltage conversion module configuredto convert the output signal produced by the linear summation moduleinto a correction voltage, and provide the correction voltage to theelectronic oscillator.

Some embodiments provide a method of controlling frequency output of anelectronic oscillator to compensate for effects of parametersexperienced by the oscillator. The method may comprise receiving firsttest data and second test data from an electronic oscillator, whereinthe electronic oscillator is subject to first and second parameters. Thefirst test data may correspond to outputs generated by the electronicoscillator while being subject to at least the first parameter over afirst range of values, and the second test data may correspond tooutputs generated by the electronic oscillator while being subject to atleast the second parameter over a second range of values. The firstparameter may be related to a temperature of at least a portion of theelectronic oscillator, and the second parameter may be related to acorrection voltage (e.g., center control voltage) provided to theelectronic oscillator. The method may further include: calculating afirst plurality of weights based at least in part on the first testdata; calculating a second plurality of weights based at least in parton the second test data; receiving from the electronic oscillator one ormore input signals corresponding to current values of the first andsecond parameters; determining, by one or more neural network processingmodules comprising one or more processors, a neural network outputsignal based at least in part on the one or more input signals and thefirst and second pluralities of weights; determining the correctionvoltage based at least in part on the neural network output signal; andproviding the correction voltage to the electronic oscillator, therebyat least partially controlling the frequency of the electronicoscillator.

In some embodiments, calculating the first plurality of weights andcalculating the second plurality of weights may be performed prior tosaid receiving the one or more input signals. The method may furthercomprise generating an intelligent seed value. In such embodiments, atleast one of calculating the first plurality of weights and calculatingthe second plurality of weights may be based at least in part on theseed value.

In some embodiments, calculating the neural network output signal maycomprise: providing the one or more input signals and the first andsecond pluralities of weights to a plurality of artificial neuronmodules of the neural network processing module, wherein at least one ofthe first plurality of weights and at least one of the second pluralityof weights are provided to each of the plurality of artificial neuronmodules; and providing a neuron output from each of the plurality ofneuron modules to a linear summer, wherein the neural network outputsignal is based at least in part on an output of the linear summer. Insuch embodiments, each of the plurality of artificial neuron modules mayutilize at least one of the plurality of weights and each of the one ormore input signals as variables of a non-linear activation function.Further, each of the plurality of artificial neuron modules may providean output that is based at least in part on a solution of the non-linearactivation function. The method may further comprise providing a biasinput to the linear summer for shifting the output of the linear summerup or down. In some embodiments, the activation function is a signoidfunction.

In some embodiments, calculating the neural network output signal maycomprise: providing the one or more input signals and the firstplurality of weights to a first plurality of artificial neuron modulesof a first neural network processing module of said one or more neuralnetwork processing modules, wherein at least one of the first pluralityof weights is provided to each of the plurality of artificial neuronmodules; providing the one or more input signals and the secondplurality of weights to a second plurality of artificial neuron modulesof a second neural network processing module of said one or more neuralnetwork processing modules, wherein at least one of the second pluralityof weights is provided to each of the second plurality of artificialneuron modules; and providing a neuron output from each of the firstplurality of neuron modules to a first linear summer and a neuron outputfrom each of the second plurality of neuron modules to a second linearsummer, wherein the neural network output signal comprises a firstneural network output signal based at least in part on an output of thefirst linear summer and a second neural network output signal based atleast in part on an output of the second linear summer.

Some embodiments provide a system for controlling frequency output of anelectronic oscillator. The system may comprise: a first input signalcorresponding to a first parameter of an electronic oscillator; a secondinput signal corresponding to a second parameter of the electronicoscillator; and a neural network processing module operatively coupledto the first and second input signals, the neural network processingmodule comprising a plurality of artificial neuron modules, wherein eachof the plurality of artificial neuron modules is configured to receivethe first and second input signals and one of one or more pre-calculatedweights and provide an output signal, for use in controlling thefrequency output of the electronic oscillator, based at least in part onthe first and second input signals and the one of the one or morepre-calculated weights. The one or more pre-calculated weights may bederived at least in part from test data that corresponds to outputsgenerated by the electronic oscillator while experiencing a first rangeof values of the first parameter and a second range of values of thesecond parameter. The first parameter may be related to a temperature ofat least a portion of the electronic oscillator. The second parametermay be related to a correction voltage provided to the electronicoscillator, wherein the correction voltage is configured to at leastpartially control the frequency output of the electronic oscillator.

In some embodiments, the system may further comprise a third inputsignal corresponding to a third parameter, wherein each of the pluralityof artificial neuron modules receives the third input signal, and theoutput signal is based in part on the third input signal. The neuralnetwork processing module may provide the output signal to a voltageconversion module configured to convert the output signal into acorrection voltage and provide the correction voltage to the electronicoscillator.

Some embodiments provide a system for providing a correction signal toat least partially compensate for effects of parameters of an electronicdevice. The system may comprise: a first neural network processingmodule comprising one or more processors and configured to receive afirst input signal corresponding to a first parameter; and a secondneural network processing module comprising one or more processors andconfigured to receive the first input signal and a second input signalcorresponding to a second parameter. Each of the first and second neuralnetwork processing modules may comprise a plurality of artificial neuronmodules. Each of the plurality of artificial neuron modules may receiveone of one or more pre-calculated weights. The first neural networkprocessing module may be configured to provide a first output signalbased at least in part on the first input signal and the one or morepre-calculated weights. The second neural network processing module maybe configured to provide a second output signal based at least in parton the first and second input signals and the one or more pre-calculatedweights.

In some embodiments, the system may further comprise one or more voltageconversion modules configured to convert the first and second outputsignals into first and second correction signals, respectively. In someembodiments, the system may further comprise a summation moduleconfigured to generate a third correction signal based on the first andsecond correction signals and provide the third correction signal to theelectronic device. The first input signal may correspond to atemperature of at least a portion of the electronic oscillator. Thesecond input signal may correspond to a correction voltage provided tothe electronic oscillator, wherein the correction voltage is configuredto at least partially control the frequency output of the electronicoscillator.

In some embodiments, the system further comprises a memory module, ananalog-to-digital converter, a digital-to-analog converter, and alow-pass filter. The memory module may be configured to store one ormore values calculated by multiplying each of the one or morepre-calculated weights by a slope control variable, and store one ormore values calculated by multiplying one or more bias inputs providedto the first and second neural network processing modules by the slopecontrol variable.

Some embodiments provide a system for compensating the effects ofcontrol voltage on an electronic oscillator to achieve a targetfrequency output from the electronic oscillator. The system maycomprise: a neural network processing module operatively coupled to aninput signal, the neural network processing module comprising aplurality of artificial neuron modules; wherein each of the plurality ofartificial neuron modules is configured to receive the input signal andone or more pre-calculated DAC values and provide an output signal, foruse in controlling the frequency output of the electronic oscillator,based at least in part on the input signal and the one of the one ormore pre-calculated DAC values; and wherein the one or morepre-calculated DAC values are derived at least in part from test datathat corresponds to outputs generated by the electronic oscillator whileexperiencing a range of control voltage values. The input signal, in oneembodiment, is related to a control voltage. The input signal, inanother embodiment, corresponds to a temperature of at least a portionof the electronic oscillator. In some embodiments, the system mayfurther comprise: a summation module configured to generate the inputsignal based on a summation of at least the control voltage and anoutput signal from a second neural network processing module, the secondneural network processing module comprising a plurality of artificialneuron modules, wherein each of the plurality of neuron modules of thesecond neural network processing module receives one of the one or morepre-calculated weights. The output signal from the second neural networkprocessing module, in one embodiment, is a correction signal based atleast in part on the one or more pre-calculated weights and an outputsignal from the electronic oscillator. The output signal from theelectronic oscillator, in one embodiment, corresponds to a temperatureof at least a portion of the electronic oscillator. The output signalfrom the electronic oscillator, in another embodiment, corresponds to anage of the electronic oscillator. The neural network processing module,in some embodiments, is further configured to receive a second inputsignal, the second input signal relating to a temperature of at least aportion of the electronic oscillator. In some embodiments, the systemfurther comprises: a voltage conversion module, wherein the voltageconversion module is configured to convert the output signal from theneural network processing module into a correction voltage, and providethe correction voltage to the electronic oscillator. In someembodiments, the system further comprises: a memory module, wherein thememory module is configured to store the one or more pre-calculated DACvalues.

Some embodiments provide for a system that provides a correction voltageto at least partially compensate for effects of parameters of anelectronic device. The system preferably comprises: a first neuralnetwork processing module comprising one or more processors configuredto receive a first input signal corresponding to a parameter of anelectronic oscillator; and a second neural network processing modulecomprising one or more processors and configured to receive the firstinput signal and a second input signal corresponding in part to acontrol voltage and an output signal from the first neural networkprocessing module; wherein each of the first and second neural networkprocessing modules comprises a plurality of artificial neuron modules,wherein each of the plurality of artificial neuron modules receives oneof the one or more pre-calculated weights, wherein the first neuralnetwork processing module is configured to provide the output signalbased at least in part on the first input signal and the one or morepre-calculated weights, and wherein the second neural network processingmodule is configured to provide a compensated control voltage based atleast in part on the first and second input signals and the one or morepre-calculated DAC values. The first input signal, in some embodiments,corresponds to a temperature of at least a portion of the electronicoscillator.

In some embodiments, the system further comprises: one or more voltageconversion modules configured to convert the output signal from thefirst neural network processing module into a correction signal. In someembodiments, the system further comprises: a summation module configuredto generate the second input signal based on the control voltage and thecorrection signal, and to provide the second input signal to the secondneural network processing module. In some embodiments, the systemfurther comprises: a memory module, an analog-to-digital converter, adigital-to-analog converter, and a low-pass filter. The memory module,in some embodiments, is configured to store one or more valuescalculated by multiplying each of the one or more pre-calculated weightsby a slope control variable, and store one or more values calculated bymultiplying one or more bias inputs provided to the first neural networkprocessing modules by the slope control variable.

Some embodiments provide for a method of compensating for effects ofcontrol voltage on an electronic oscillator to achieve a targetfrequency output from the electronic oscillator. The method preferablycomprises: receiving test data from an electronic oscillator, theelectronic oscillator being subject to a test control voltage, whereinthe test data corresponds to outputs generated by the electronicoscillator while being subject to the test control voltage over a rangeof test control voltages; calculating a plurality of DAC values at leastin part on the test data; providing a current control voltage to aneural network processing module including one or more processors;providing the plurality of DAC values to the neural network processingmodule; using the neural network processing module to calculate a neuralnetwork output signal based at least in part on the current controlvoltage and the plurality of DAC values; determining a correctionvoltage based at least in part on the neural network output signal; andproviding the correction voltage to the electronic oscillator to atleast partially control the frequency of the electronic oscillator. Insome embodiments, calculating the first plurality of DAC values isperformed prior to said receiving the current control voltage.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are depicted in the accompanying drawings forillustrative purposes, and do not limit the scope of the inventions. Inaddition, various features of different disclosed embodiments can becombined to form additional embodiments, which are part of thisdisclosure.

FIG. 1 is a schematic diagram of an embodiment of a neural network.

FIG. 2 is a schematic diagram of an embodiment of a neuron.

FIG. 3A is a schematic diagram of an embodiment of a neuron.

FIG. 3B is a schematic diagram of an embodiment of a neuron.

FIG. 4 is a graph of an embodiment of a neuron response with preselectedinitial weights.

FIG. 5 is a graph of an embodiment of a curve fit of a parabola.

FIG. 6A is a graph showing an embodiment of neuron division over atemperature range for training purposes.

FIGS. 6B-6D provide graphs representing potential embodiments ofsolutions for portions of the sample fit shown in FIG. 6A.

FIG. 7 is a schematic diagram of an embodiment of a neural network.

FIG. 8 is a schematic diagram of an embodiment of a neuron.

FIG. 9A is a graph of embodiments of a sigmoid curve.

FIG. 9B is a graph of embodiments of a sigmoid curve.

FIG. 9C is a graph of embodiments of a sigmoid curve.

FIG. 10 is a schematic diagram of an embodiment of a neural network.

FIG. 11 is a graph showing test data relating to oscillator temperatureresponse.

FIG. 12 is a graph showing test data relating to error of a polynomialfunction generator.

FIG. 13 is a graph showing test data relating to performance of anembodiment of neural network frequency controller.

FIG. 14 is a graph showing test data relating to error of a neuralnetwork frequency controller.

FIG. 15 is a block diagram of an embodiment of a neural networktemperature compensation system.

FIG. 16 is a schematic diagram of an embodiment of a neural networktemperature compensation system.

FIG. 17 is a block diagram of an embodiment of a neural networktemperature compensation hardware system.

FIG. 18 is a graph showing data relating to oscillator temperatureresponse.

FIG. 19 is a graph showing data relating to performance of an embodimentof neural network temperature-compensation frequency controller.

FIG. 20 is a graph showing data relating to performance of a5^(th)-order polynomial compensated oscillator.

FIG. 21 is a graph showing data relating to performance of an embodimentof a neural network temperature-compensation frequency controller.

FIG. 22 is a graph showing data relating to two compensation systems.

FIG. 23 is a block diagram of an embodiment of a neural networktemperature and trim effect compensation system.

FIG. 24 is a schematic diagram of an embodiment of a neural networktemperature and trim effect compensation system.

FIG. 25 is a graph showing information relating to trim effect of anelectronic oscillator.

FIG. 26 is a graph showing information relating to performance of aneural network temperature and trim effect compensation system.

FIG. 27 is a block diagram of an embodiment of a neural networkfrequency-drift compensation system.

FIG. 28 is a schematic diagram of an embodiment of a neural networkfrequency-drift compensation system.

FIG. 29 is a block diagram of an embodiment of a neural networkacceleration and vibration compensation system.

FIG. 30 is a block diagram of an embodiment of a neural network pressurecompensation system.

FIG. 31 is a block diagram of an embodiment of a neural networkhysteresis compensation system.

FIG. 32 is a block diagram of an embodiment of a neural networkoven-controller system.

FIG. 33 is a block diagram of an embodiment of a neural networkdouble-oven controller system.

FIG. 34 is a graph illustrating a three dimensional sigmoid sheet,according to an example embodiment.

FIG. 35 is a schematic diagram illustrating a two-input network for trimeffect compensation, according to an example embodiment.

FIG. 36 is a block diagram illustrating an artificial neural networktrim effect compensation system, according to an example embodiment.

FIG. 37 is a block diagram illustrating example hardware used forimplementing an artificial neural network trim effect compensationsystem, according to an example embodiment.

FIG. 38 is a graph illustrating the deviation in an electronicoscillator before implementing trim effect compensation, according to anexample embodiment.

FIG. 39 is a graph illustrating the deviation in an electronicoscillator after implementing trim effect compensation, according to anexample embodiment.

FIG. 40 is a graph illustrating the deviation in an electronicoscillator before implementing control voltage linearity compensation,according to an example embodiment.

FIG. 41 is a graph illustrating the deviation in an electronicoscillator before implementing control voltage linearity compensation,according to an example embodiment.

FIG. 42 is a block diagram illustrating an artificial neural networkcontrol voltage linearity compensation system, according to an exampleembodiment.

FIG. 43 is a graph illustrating a control voltage map of ADC and DACvalues against a desired output frequency, according to an exampleembodiment.

FIG. 44 is a graph illustrating a control voltage map of ADC valuesagainst DAC settings for target frequency, according to an exampleembodiment.

FIG. 45 is a graph illustrating the deviation in an electronicoscillator after implementing control voltage linearity compensation,according to an example embodiment.

FIG. 46 is a block diagram illustrating an artificial neural networkcontrol voltage linearity compensation system, according to an exampleembodiment.

FIG. 47 is a block diagram illustrating an artificial neural networkcontrol voltage linearity compensation system, according to an exampleembodiment.

FIG. 48 is a block diagram illustrating an artificial neural networkcontrol voltage linearity compensation system, according to an exampleembodiment.

FIG. 49 is a block diagram illustrating an artificial neural networkcontrol voltage linearity compensation system, according to an exampleembodiment.

FIG. 50 is a block diagram illustrating an artificial neural networkcontrol voltage linearity compensation system, according to an exampleembodiment.

FIG. 51A is a graph illustrating the deviation in an electronicoscillator before implementing control voltage linearity compensation,according to an example embodiment.

FIG. 51B is a graph illustrating the deviation in an electronicoscillator after implementing control voltage linearity compensation,according to an example embodiment.

DETAILED DESCRIPTION

Certain embodiments of neural networks are suitable for functionapproximation, or curve fitting applications, and for application in thedomain of frequency control. For example, an artificial neural network(referred to herein as “ANN” or “NN”) may be used in the control of anelectronic oscillator, or other frequency standard device. In someembodiments, an ANN is utilized in the correction of crystaloscillators. Frequency standard devices, such as electronic oscillators,are often subject to certain short-term instabilities, which may affectthe performance of the system. Such instabilities can be caused by,among other things, temperature fluctuations, vibration, noise due toactive and passive circuit components, fluctuations at resonatorinterfaces, magnetic field, trim effect, pressure, warm-up of thedevice, thermal distribution due to physical orientations of the device,aging, or other causes. Compensation for one or more of these parametersmay be achieved through the design of an appropriately configured ANN.Furthermore, the principles disclosed herein may be applicable to areasbeyond frequency control, and discussion herein in the context offrequency control is not limiting on the scope of this disclosure.

Although certain embodiments and examples are disclosed below, inventivesubject matter extends beyond the specifically disclosed embodiments toother alternative embodiments and/or uses and to modifications andequivalents thereof. Thus, the scope of the invention is not limited byany of the particular embodiments described below. For example, in anymethod or process disclosed herein, the acts or operations of the methodor process may be performed in any suitable sequence and are notnecessarily limited to any particular disclosed sequence. Variousoperations may be described as multiple discrete operations in turn, ina manner that may be helpful in understanding certain embodiments;however, the order of description should not be construed to imply thatthese operations are order dependent. Additionally, the structures,systems, and/or devices described herein may be embodied as integratedcomponents or as separate components. For purposes of comparing variousembodiments, certain aspects and advantages of these embodiments aredescribed. Not necessarily all such aspects or advantages are achievedby any particular embodiment. Thus, for example, various embodiments maybe carried out in a manner that achieves or optimizes one advantage orgroup of advantages as taught herein without necessarily achieving otheraspects or advantages as may also be taught or suggested herein.

Conditional language used herein, such as, among others, “can,” “might,”“may,” “e.g.,” and the like, unless specifically stated otherwise, orotherwise understood within the context as used, is generally intendedto convey that certain embodiments include, while other embodiments donot include, certain features, elements and/or states. Thus, suchconditional language is not generally intended to imply that features,elements and/or states are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without author input or prompting, whether thesefeatures, elements and/or states are included or are to be performed inany particular embodiment. The terms “comprising,” “including,”“having,” and the like are synonymous and are used inclusively, in anopen-ended fashion, and do not exclude additional elements, features,acts, operations, and so forth. Also, the term “or” is used in itsinclusive sense (and not in its exclusive sense) so that when used, forexample, to connect a list of elements, the term “or” means one, some,or all of the elements in the list.

Some embodiments include neural networks with interconnected groups ofartificial neurons, referred to herein as neurons, and may providefunctionality to process often complex information. Additionalinformation about neural networks may be found in “Neural Networks: AComprehensive Foundation.” Simon Haykin, (2d ed., 1998). The specificconnection architecture used helps to define functionality of aparticular neural network. Neurons within a neural network are oftenorganized into layers, as depicted in FIG. 1. While the modelillustrated in FIG. 1 consists of nine neurons organized into threelayers (organized by vertically arranged columns of neurons), ANN's maycomprise any desirable number and/or arrangement of neurons, includingany number of layers. The embodiment depicted in FIG. 1 comprises aninput layer. The input layer may not include full neurons, but rathermay include values in a data record, for example, that constitute inputsto the next layer of neurons. The activity of input neurons mayrepresent raw information that is fed into the neural network.

Neural Network Structure

In some embodiments, an ANN may include one or more “hidden” layers. Asshown in FIG. 1, the second layer may be considered a hidden layer insome embodiments. Unlike the input and output layers, the hidden layermay not be connected externally. A hidden layer may be used to recode,or provide a representation for, the inputs provided by the input layer.The activity of each hidden unit, or neuron, may be determined by theactivities of the input neurons and the weights applied to theconnections between the input and hidden neurons. In some embodiments,there are multiple hidden layers. Some embodiments do not include anyhidden layers. Various other layers may be included in place of, or inaddition to, one or more hidden layers. For example, an ANN may includea linear summer layer, which provides as output the sum of the inputs tothe layer, possibly offset or modified in some way.

In the system of FIG. 1, the far-right layer is an output layer. Asingle sweep through the network from left to right results in theassignment of a value to each output node. The term “layer” is usedherein according to its broad and ordinary meaning. For the purpose ofdescription, “layer” may be used to refer to any stage of a neuralnetwork, including input and/or output stages, or may refer only tointernal or hidden stages. The system depicted in FIG. 1 represents afeedforward system. “Feedforward” may refer to systems where connectionsbetween neurons do not form cycles or loops; information moves in onedirection, from the input nodes, through any inner/hidden nodes, and tothe output nodes. However, the principles described herein may beapplicable to any suitable network configuration, such as a recurrentneural network.

The individual neurons that make up a neural network can be based on anynumber of models. At a high level, neurons often take on the basicconfiguration depicted in FIG. 2. As illustrated in FIG. 2, a number ofinputs, each of which is associated with a weight, ‘w,’ are fed into aprocessing module 220 which uses the inputs to calculate a singleoutput. In some embodiments, a neuron may calculate, at least in part,an output by summing its inputs, or by summing inputs multiplied by oneor more weights. In some embodiments, weights are floating point numbersthat can be dynamically adjusted.

Activation Functions

The processing module 220, which acts as a logical operator, is oftenreferred to as a decision, or activation, function. FIG. 3A provides amore detailed diagram of an embodiment of a neuron model that may beused in frequency control or other applications. In some embodiments,the purpose of the activation function is to limit the amplitude of theoutput to some finite value. The model depicted in FIG. 3A is referredto as a perceptron model. As is shown in FIG. 3A, input vectors arescaled with individual synaptic weights ω₁ through ω_(n), and fed into alinear summation module. The inputs are linearly summed together alongwith a bias term, ‘b’. In FIG. 3A, the resulting linear sum is denotedas ‘z’. The linear sum is then fed through an activation function toproduce an output ‘y.’

An activation function can have any desirable design. Many practicalactivation functions exist, and different types of issues/problems maycall for solutions based on different activation functions. Table Abelow provides a list of commonly-used activation functions. However,the list of Table A is not comprehensive, and other activation functionsnot listed may be practical or desirable, depending on characteristicsof the relevant system. For example, a system could incorporate one ormore polynomial activation functions, such as first, second, third,fourth, fifth, or higher-order polynomial activation functions.Furthermore, a system may incorporate a combination of different typesof activation functions.

TABLE A Activation Function Output Equation Uni-Polar either a 0 or 1output 1 if z ≧ 0 Binary y = {open oversize brace} 0 if z < 0 Bi-Polareither a −1 or 1 output 1 if z ≧ 0 Binary y = {open oversize brace} −1if z < 0 Linear a linear scale of the y = w_(l)z summation applied wherew_(l) is the gain of the linear activation function Uni-Polar Sigmoidbetween 0 and 1 that follows the sigmoid shape factor$y = \frac{1}{1 + {\mathbb{e}}^{- {\alpha{(z)}}}}$ the output iscontinuous and differentiable Bi-Polar Sigmoid between −1 and 1 thatfollows the sigmoid shape factor$y = {\frac{2}{1 + {\mathbb{e}}^{- {\alpha{(z)}}}} - 1}$ the output iscontinuous and differentiable

Frequency control using ANN techniques, such as ANN correction ofcrystal oscillators, may be suitable for a curve fitting application.For example, ANN applications may provide frequency correction forAT-cut crystal oscillators, such as AT-cut strip crystals manufacturedby Statek Corporation, Orange, Calif. As may be known by those havingordinary skill in the art, approximation theory states that any signalcan be constructed from a sum of any other signal. A special case ofapproximation theory provides that any signal can be broken up into asum of sine and cosine signals. In view of approximation theory, neuronoutput signals may be summed together to fit, or approximate, a desiredsignal, such as a frequency curve. However, certain activation functionsmay be better suited than others for fitting or approximating afrequency curve.

In some embodiments, a linear activation function is used for fitting afrequency curve. However, as a linear activation function provides alinear output signal, it may be difficult to fit a non-linear frequencycurve using such an activation function. In some embodiments, an ANNutilizing a linear activation function is used in combination with oneor more other, possibly less complex, controllers, such as an analogcircuit or other controller. The combined usage may allow for closerapproximation of a frequency curve.

In certain ANN embodiments designed for frequency control, it may bedesirable to use a uni-polar sigmoid activation function. However, it iscontemplated that any of the above-listed activation functions, or anyother suitable activation function, may be used in frequency controlapplications. As shown in FIGS. 9A-9C, a sigmoid activation function,unlike linear or binary activation functions, includes beveled shape ateach end. The beveled shape of the sigmoid function may allow forsmoother curve fitting with fewer elements.

FIG. 3A illustrates a schematic diagram of an embodiment of a neuron. Asshown in FIG. 3A, in some embodiments, an input vector including inputsX₁ through X_(n) is multiplied by the individual synaptic weights ω₁through ω_(n), respectively. An additional bias signal, b, is alsoprovided to the linear summation module. In some embodiments, the bias bhas one or more of its own weights ω_(b). These products are summedtogether before being provided to the activation function (e.g., Φ).Equations (1) and (2) show the neuron in mathematical terms.Z=(Σ_(i=1) ^(n)ω_(i) X _(i))+ω_(b) b  (1)Y=Φ(Z)  (2)

For simplicity, the diagram of FIG. 3A may be condensed so that thesummation and the activation function are both contained in a singlemodule (e.g., neuron bubble) as shown in FIG. 3B. Furthermore, while theembodiment of FIG. 3A shows the linear summer module positioned in thedata path before the activation function, such functional modules may beconfigured in any suitable logical or physical order.

With respect to uni-polar sigmoid activation functions, the output valueof the function may range from “0” to “1.” Therefore, it may bedesirable for a data set to be modeled to output values between “0” to“1” so that the ANN can properly fit the data. The weights may beinitialized with random numbers. Alternatively, based on anunderstanding of sigmoid functions, starting values may be selected topre-shape the individual sigmoid functions so that they are at leastsomewhat “close” to where they need to be, making the training of thenetwork potentially easier. These concepts are discussed in greaterdetail below.

Intelligent selection of starting values may be useful since gradientreduction methods may be prone to getting stuck in local minima. Forexample, if one would like to fit a parabola (y=x²) using the ANNstructure described below with reference to FIG. 7 (e.g., two neurons inthe input layer), the initial values could be intelligently chosen, asshown in Table B. Table B provides neuron values with preselectedinitial weights. This may shift the neurons y₁ and y₂ left and right,producing the individual responses shown in FIG. 4.

TABLE B Variable y₁ y₂ Y_(out) α 1 −1 1 ω 1 1 1 b −6 6 0

With these starting values, the method of least squares may obtain asolution for the network output as shown in FIG. 5, wherein the outputsof Neuron 1 and Neuron 2 are combined to approximate the shape of theparabola. The values for the relevant network weights are provided belowin Table C.

TABLE C Variable y₁ Y₂ Y_(out) α 0.756 −0.756 1 ω 0.871 0.871 1 b −5.9795.979 0

The above example illustrates that, in some embodiments, an ANN may beused to approximate more complex functions than its individual neuronscan themselves imitate.

While the computational power of a single neuron may be insufficient tofunction as an accurate curve-fitting operator in certain circumstances,as the number of neurons increases, the computational power of thesystem may also increase. The computational power of the system may beat least partially governed by the connectivity between the neurons. Inan embodiment comprising a single neuron with two inputs, x₁ and x₂, anda bias, b, the solution may include the calculation of three distinctweights. However, as illustrated in FIG. 7, if another neuron, Ø₂, wereto be added to the two-input system, a third neuron, Ø₃, may also benecessary to tie the two solutions (y1, y2) together into one output(y3). Such a configuration may require nine coefficients to be solved.As additional neurons are added to the system, the number of weights mayincrease significantly, thereby increasing the computational complexityof the system. As shown in FIG. 7, the output layer has inputs y1, y2that are the outputs of the neurons from the previous layer. Thisstructure can produce an output y₃ that is more complex than any of theindividual neurons themselves can create. While, as discussed above, anANN may include hidden layers, for the purposes of curve fitting, in maynot be necessary to include a hidden layer.

Neural Network Training

Once the connection architecture and activation function of an ANN havebeen chosen, in some embodiments, the input weights must be “trained” sothat the output classifies the input signal to the desired output. Forexample, the network may be trained using supervised learning orunsupervised learning. In some embodiments, the weights are trainedusing backpropagation, a supervised learning method.

In some embodiments, a signal from a reference oscillator is fed backinto the training system, and an error signal representing thedifference between the reference signal and the output signal isanalyzed in view of one or more temperature inputs. In otherembodiments, no reference signal is used. Training may involvesubjecting the oscillator unit to a range of temperatures and recordingoutput measurements at discrete intervals. For example, readings may betaken in 1° C. increments, and may be taken over a temperature range. Insome embodiments, temperature readings are taken over a range ofapproximately 0-7° C. In some embodiments, readings are taken over arange of approximately 5° C., 10° C., 15° C., or 20° C., or more. In yetanother embodiment, readings are taken over a range of more than 120°C., such as over the range of approximately −45° C. to 90° C., as shownin the embodiments of FIGS. 6A-6D. The test data may be used by thesystem to calculate the input weights to the system. In someembodiments, a computer processor solves for the required weights andbias, or biases, to achieve a curve that matches the crystal frequencyover temperature as closely as possible. Such calculation may beimplemented using a least-squared solution, and may involve agradient-reduction search. For example, the squared error at eachmeasured temperature point could be summed together, with the minimumsquared error over the temperature range being solved for, as well asthe corresponding weights that produce such error.

With respect to a particular task or ANN application, one trainingmethod may provide advantages over another. In some embodiments, weightsof an ANN may be randomly seeded (“Method 1”). However, for reasonsdisclosed below, such methodology may not provide ideal performanceand/or efficiency. For example, with respect to a sigmoid-basedactivation function, known characteristics of such functions may beleveraged to intelligently select starting values that allow forpotentially better and/or more efficient solutions to be obtained thanby using random seeding (“Method 2,” discussed in greater detail below).Furthermore, preselecting “areas of interest” for individual neurons mayalso improve selection/performance.

Some embodiments of ANN training methods include taking the entirenetwork as a whole and applying the solution algorithm to an associatedmatrix. However, such solution algorithms may have a tendency to getstuck in local minima. Furthermore, the ANN and solution space itselfmay result in adverse effects. For example, as the solution space growsin complexity, the number of neurons needed to fit the curve generallyincreases, thereby increasing the number of weights that are iterated inthe solution. Such increase can create additional local minima that canpotentially hamper the solution algorithm.

FIG. 6A is a graph showing how neurons may initially be divided over aset of sample data. As shown, in some embodiments, different neurons aretrained over different initial temperature ranges. From the initialranges for each neuron, it may be possible to preselect values that setthe slope, delay, and/or amplitude such that an initial solution of theneuron is relatively close to, or within an acceptable proximity of, afinal solution associated with the neuron. From these initial solutions,each neuron can be solved over its associated temperature range/space,which may result in improved curve fit over the respective range ofinterest.

FIGS. 6B-6D provide graphs representing potential solutions for portionsof the sample fit shown in FIG. 6A. For example, FIG. 6B shows anexample curve fit after gradient reduction has been performed on a firstneuron (e.g., Neuron 1). As shown, the curve fits the relevant datarelatively well over an area of interest, but is uncontrolled outsidethe area of interest. In some embodiments, the contribution outside thearea of interest is advantageously near “0.” This may make remainingcalculations for the solution relatively simpler. FIG. 6C shows anexample curve fit after gradient reduction has been performed on asecond neuron (e.g., Neuron 2). Similarly to the curve of Neuron 1, thecurve of FIG. 6C fits the relevant data relatively well over thecorresponding area of interest, as defined in FIG. 6A. Furthermore,outside of the area of interest, the curve's contribution is near “0” tothe left and a somewhat close fit to the right of the area of interest.

FIG. 6D provides a graph representing a potential solution that is basedon a summation of the curves of Neurons 1 and 2, as described above,after such summation has been run through a gradient reduction to reduceits sum squared error. The combined solution for Neuron 1 and Neuron 2over the combined areas of interest for Neuron 1 and 2 substantiallymatches the fit data.

The example areas of interest of FIGS. 6B-6D provided the benefit ofhaving solutions around local minima at a deviation close to 0.Furthermore, the associated data set shown may be considered arelatively simple data set. These factors may allow for individualsolutions to be near “0” in certain regions outside of the area ofinterest, which may make the summation solution much easier tocalculate. In more complex data sets, such may not be the case, and theinteractions may not be as easy to visualize.

A comparison of performance of an ANN frequency controller trained usingan embodiment of Method 1 to a controller trained using an embodiment ofMethod 2 is outlined below in Table D. As stated above, Method 2corresponds to a method in which neurons are intelligently placed andseeded, and the individual neurons are trained and then the sums aretrained. The first entry in Table D is labeled Method 1, and shows anexample sum-squared error and peak-to-peak residuals. The second entrycorresponds to Method 1 calculated for the whole matrix, which issimilar to the method described above, but the whole matrix is solvedafter the intelligent placement and seeding. In some embodiments, noindividual solutions or walking sums are trained. The solution may havealmost three times as much residual error. Finally, the last 5 entriescorrespond to Method 2, wherein random seeds are used and the entirematrix is solved at once. One can see that none of the 5 solutions areeven close to the Method 1 solutions, and they generally tend to fallinto the same local minima producing nearly duplicate results around11.27 and 4.5 for residual error.

TABLE D Method Sum squared Error Residual pk-to-pk (ppm) Method 10.065301276 0.120511 Method 1- whole matrix 0.367575503 0.349724 Method2 - whole matrix 1 391.7005882 11.27486 Method 2 - whole matrix 293.00323905 4.58059 Method 2- whole matrix 3 391.7005899 11.27485 Method2- whole matrix 4 101.3273131 4.523202 Method 2 - whole matrix 5101.6971708 4.447988

Method 1 may provide curve fitting functionality to the level ofaccuracy that is needed for the ANN compensation to be effective. On theother hand, using Method 2 may render the ANN compensation unusable insome embodiments. By using more neurons than shown above, the residualpeak-to-peak error of this solution can be driven down to under 0.020ppm in some embodiments. This level of curve fitting may not be feasiblewith the industry standard 5^(th)-order polynomial.

FIG. 8 illustrates a block diagram of a neuron that may be used in acurve-fitting application. In some embodiments, the activation functionassociated with the neuron is a uni-polar sigmoid activation function.The neuron may include a single weight, ω₁, and two biases, b and a (notshown). Such a configuration may be computationally beneficial in thatit may comprise fewer values to solve for than multi-input,multi-connected networks. In an embodiment configured for oscillatortemperature compensation, the input ‘x’ may represent temperature, forexample. In some embodiments, as shown in FIG. 8, the bias is fixed at avalue of ‘1’. In curve-fitting embodiments, the output, ‘y’, may be adesired signal correction corresponding to operation of an electronicoscillator. This output level may be converted to a correction voltageusing the mV/ppb sensitivity of the oscillator, or by any otheravailable mechanism.

In some embodiments, the activation function of the neuron depicted inFIG. 8 may be a uni-polar activation function conforming to thefollowing equation:

$\begin{matrix}{\varnothing = \frac{1}{1 + {\mathbb{e}}^{- {\alpha{({{\omega\; x} + b})}}}}} & (3)\end{matrix}$

The activation function can include at least three variables, themodification of which will alter the shape and/or position of the outputsignal. In some embodiments, the values of one or more of the variablesα, ω₁, and b are iteratively calculated during the training phase of theANN. In some embodiments, one or more of the variables is set to aconstant value of either 0 or 1.

FIG. 9A is a graph that demonstrates the effect that the weight ω₁ hason the sigmoid activation function output. As is shown in the graph,adjustment in the value of ω₁ adjusts the amplitude of the signal,wherein a higher value of ω₁ generally results in greater amplitude.This effect may be more clearly illustrated when the bias, b, isnon-zero, and α is something other than unity.

FIG. 9B is a graph that demonstrates the effect that the bias weight, b,has on the sigmoid activation function output. As is shown in the graph,adjustment of the value of b, which may be temperature in the case ofoscillator curve fitting, shifts the sigmoid curve to the left or right.As shown, a negative value of b may generally cause a shift to theright, while a positive value of b causes a shift to the left. In anapplication where the input is temperature, as shown in the figure,modification of the value of b may in effect shift an individual sigmoidfunction's output in response to an input temperature.

FIG. 9C is a graph that demonstrates the effect of a on the sigmoidactivation function output. As is shown in the graph, the slope of thesigmoid curve is controlled by the value of α. By changing the sign ofα, the sign of the slope of the activation function output may also bechanged (e.g., from positive slope to negative slope). In someembodiments, the slope of the sigmoid is set at either 1 or −1 to obtainthe direction of slope desired, and the ω₁ and b weights are solved for.This approach may make the solution of the system easier to obtaincomputationally. However, such an approach may also increase the numberof neurons required to obtain a satisfactory fit of the curve. In someembodiments, rather than setting α to a constant value, the value of αis solved for. Iteratively solving for a may help reduce the number ofneurons necessary to fit a curve of interest. By controlling the slopeof a sigmoid curve, rather than using a constant value for α, it may bepossible to achieve a curve fit that otherwise would require summingmultiple curves together, in this case sigmoid-shaped curves.Furthermore, this approach may provide more versatility from each neuronin the network, increasing the precision of the system.

FIGS. 9A-C show that, in certain three-variable embodiments, the sigmoidamplitude, slope, and delay (left/right position) can be controlled.Multiple neurons may be summed together to create more complexfunctions. In regard to the application of oscillator compensation,there may also be a need or desire to shift the sigmoid up and down as afrequency adjust. In some embodiments, if the ANN structure in FIG. 7 isused, a user may have no ability to shift the sigmoid functions up anddown. For example, the output layer of the network, Ø₃, may be anothersigmoid shaped neuron, which may not allow for up/down shifting of thesignal. To address this problem, in some embodiments, the ANN structureof FIG. 7 may be altered. The output neuron may be replaced with alinear summation module, which may have its own input weights, outputweights, and/or dedicated bias. Such an embodiment is described below infurther detail with respect to FIG. 10. In such an embodiment, thelinear summer bias value ‘b’ may provide the network with a frequencyadjust (up/down shift).

FIG. 10 illustrates an embodiment of an ANN architecture that may beused for curve fitting applications. In some embodiments, the system ofFIG. 10 is used in temperature compensation of electronic oscillators,such as crystal oscillators. The network includes a layer of neurons,Ø₁, Ø₂ . . . Ø_(n), each connected to two inputs, one of which may betied to a value of 1. The network shown in FIG. 10 uses a single inputvoltage X (which may represent temperature). After being trained, thenetwork may provide a signal output corresponding to the deviation inppm. Certain data modeled herein was normalized to meet the “0” to “1”level of the sigmoid functions in the network, and therefore it may benecessary for such data to be decompressed. A weight may be applied tothe output in order to decompress the model and provide corrected outputdeviation. Furthermore, the output itself may be the deviation. Tocreate a compensation voltage, the signal may be converted into avoltage that will steer the oscillator in the opposite direction of thedeviation. FIG. 15, discussed in further detail below, shows a blockdiagram of an oscillator system including gain blocks that may be usedto achieve this goal.

The network may include any desirable number of neurons in the firstlayer. For example, the network may include only a few neurons in thefirst layer, or may include 25, 30, 33, 40, or any other number ofneurons. The number of neurons chosen may affect the ability of thesystem to effectively fit the desired curve. For example, addingadditional neurons to the network may increase the degrees of freedom ofthe system. In some embodiments, each of the inputs to the variousneurons in the network is multiplied by an independently calculatedweight (e.g., ω₁, ω₂ . . . ω_(n)). The output of each of the neurons(Y₁, Y₂ . . . Y_(n)) in the network is provided to a second layer thatcomprises a linear summation module 930.

In some embodiments, the network of FIG. 10 is governed by the followingequations:

$\begin{matrix}{{\varnothing = \frac{1}{1 + {\mathbb{e}}^{- {\alpha{(z_{k})}}}}};} & (4)\end{matrix}$ε_(n)=ω_(n)(x)+ω_(0n);  (5)Y _(n+1) =Y _(1ω11) +Y _(2ω21) + . . . +Y _(nωn1) +b;  (6)

wherein α=slope of the sigmoid curve.

The system embodied in FIG. 10 may provide a number of advantages overcertain other embodiments. As discussed above, α may be iterativelycalculated in the solution matrix. This may reduce the number of neuronsneeded to obtain a precise fit. A reduced number of neurons may aid inreducing the footprint and power consumption of a device implementingthe embodied system. Furthermore, the network illustrated in FIG. 10includes as a second layer a linear summer 930 with independent weightsand bias, as opposed to a neuron operated by an activation function. Insome embodiments, the system includes a hidden layer with the sameactivation function as the input layer. In some embodiments, the outputlayer does not include a linear summer, and may include anotheractivation function. As mentioned above, the bias for the first layer ofneurons imposes a left/right shift, which may represent temperature in atemperature compensation application. While the system of FIG. 10 mayinclude a temperature input from a single temperature sensor, someembodiments employ the use of multiple temperature sensors. Such asystem may be desirable in cases where different portions of a unitexperience different temperatures, such that more accurate temperaturedetermination is possible with multiple sensors. However, additionaltemperature sensors would require more inputs to the system, affectingthe complexity of the system, while possibly providing greater accuracy.

The bias, b, for the linear summer, imposes an up/down shift which mayperform a frequency adjustment for the entire curve. Without frequencyadjustment, it may be difficult to obtain a good fit because thenetwork, while mimicking the shape, may not be able to “overlay” thecurve in a desired shape.

In some embodiments, an ANN for use in frequency control may beimplemented with a hardware model. For example, an ANN frequencycontroller may be implemented in a custom application-specificintegrated circuit (ASIC). In some embodiments, an ANN is built usingdiscrete circuit components.

Neural network frequency control may be implemented using an idealizedcomputer model. FIGS. 11-14 comprise example data demonstratingpotential performance of an embodiment of an ANN frequency controllervis-à-vis the performance of a system using a curve-fitting polynomialfunction generator. With respect to the ANN system, crystal data wassubmitted to the network to be fitted. The same data was also submittedto an idealized 5^(th) order polynomial function. FIG. 11 shows the dataset and the resulting 5^(th) order fit. This data was selected inparticular because it was from a discrete oscillator having acharacteristic general inability to compensate certain crystals due todistortion. The curve with data points represented by diamondsrepresents the raw data, while the curve with data points represented bysquares represents the 5^(th) order fit.

FIG. 12 shows the potential residual error between the 5^(th) order fitand the raw data. The magnitude of the error, particularly at theexample data points at approximately 2.6° C. and 3.3° C., may beunacceptable for real-world oscillator applications. By certainstandards, a residual error of roughly +/−10.5 ppm may effectivelydesignate a tested crystal as a throw away crystal, and may fall shortof certain application requirements, such as, for example, a requirementthat residual error not exceed +/−0.5 ppm. This same data was submittedto an embodiment of an ANN constructed in accordance with the embodimentdescribed in connection with FIG. 10. FIGS. 13 and 14 show the fit ofthe neural network controller to the raw data, and residual error,respectively. The ANN embodiment tested comprised six neurons. As isshown in FIG. 14, the residual error of the ANN fit may be approximately+/−0.3 ppm. Therefore, the error from an idealized 5^(th) order curvefit may be reduced by approximately a factor of 5 through use of an ANNtemperature compensation system. Furthermore, the error of the ANNcontroller meets application requirements for stability requiring errorto be within +/−0.5 ppm.

Additional Embodiments

Some of the embodiments disclosed herein may be suited fortemperature-compensated crystal oscillators (TCXO). Furthermore,principles disclosed herein are applicable to compensation for a host ofother problems. In some embodiments, ANN's are utilized as secondarycompensation of oven-controlled crystal oscillators (OCXO). Applicationof secondary neural network temperature compensation to an OCXO mayallow for performance levels approximating double oven performance froma single oven. Other non-limiting example applications may include TCXOwarm-up compensation, TCXO trim effect compensation, OCXO transientresponse compensation, and others.

The use of an ANN to compensate for various parameters in oscillatorperformance may be realized as a temperature-compensated crystaloscillator (“TCXO”), an oven-controlled crystal oscillator (“OCXO”), amicrocomputer compensated crystal oscillator (“MCXO”), or any otherfrequency control application, such as a micro-electronic mechanicalsystem oscillator (“MEMS”) or chip-scale atomic clock (“CSAC”). Themeasurement of various parameters associated with a frequency standarddevice may be accomplished with a single electrical sensor, a pluralityof electrical sensors, or by other means. A system in accordance withthis disclosure could be constructed as an ASIC or Discrete AnalogHardware, or system functions may be synthesized through amicroprocessor, or other means.

In addition to temperature compensation, ANN's may be used in a widerange of curve fitting applications. A number of embodiments of suchsystems are described below. However, the particular embodimentsdiscussed do not limit the applicability of neural network controllersystems. The example embodiments below utilize a uni-polar sigmoidactivation function, Ø, for individual neurons. The uni-polar sigmoidfunction, as discuss above, is defined as:

$\begin{matrix}{\varnothing = \frac{1}{1 + {\mathbb{e}}^{- {\alpha{({{\omega\; x} + b})}}}}} & (7)\end{matrix}$

However, in certain multi-input neuron embodiments, the (ωx+b) portionof the above equation is a sum of all inputs times all weights, as willbe illustrated below. While the uni-polar sigmoid function is discussedherein, any suitable activation function may be used.

FIG. 15 provides a block diagram of an ANN temperature compensationsystem, while FIG. 16 shows a schematic diagram of such an ANN system.In some embodiments, the system may include a quartz crystal oscillator,or any other suitable oscillator. For example, suitable oscillators mayinclude oscillators that are tunable via a voltage control, such asvoltage-controlled oscillators (VCO's). The voltage control may beexternally controllable by a user, or may be controlled internally.

FIG. 17 provides a block diagram illustrating a hardware embodiment ofthe ANN temperature compensation system shown in FIG. 15. The trainedweights for the ANN may be stored in an EEPROM, or other type of memorymodule. Each neuron may include three or more variables that are to betrained (e.g., ω, b, and α). However, as shown in Equation (3), α may bedistributed in the quantity z_(k). After training, in some embodiments,distribution can be implemented such that what is stored in the EEPROMare the quantities αω and αb. This may reduce the storage requirementsfor each neuron down to two quantities. The network may also have tostore values for b and/or ω_(out). Therefore, for example, an embodimentcomprising 25 neurons for primary compensation may require approximately52 values to be stored, whereas a 33-neuron embodiment may requireapproximately 68 values to be stored.

In some embodiments, the microprocessor shown in FIG. 17 samples atemperature sensor using an ADC. Output from the ADC may then be inputto the microprocessor which calculates the ANN's response, and theoutput is converted back into a voltage signal using a DAC. In certainhardware embodiments, approximately seven corrections can be applied tothe oscillator per second, advantageously providing tracking under fasttemperature-ramp conditions. A low-pass filter may be incorporated toprevent the output frequency from having a step function response to thecorrections.

In the embodiment of FIG. 17, the temperature sensor may reside insidethe oscillator unit as near to the resonator as possible to reducetemperature gradients between the sensor and the resonator. In someembodiments, the temperature sensor is the sole input to the ANN. TheANN may learn the oscillator's temperature characteristics through atraining process and allow a correction voltage to be created that isthen fed into the control voltage input (EFC) pin of the oscillator. Thecorrection voltage may be summed in with a user's desired EFC voltage inthis embodiment.

In the schematic diagram of FIG. 16, ‘X’ may represent a temperaturesensor input and ‘b’ and C may represent some constants which can be anyreal number. The respective weights may be actively solved for. In someembodiments, for ease of calculation, they may be set to 1. It is notnecessary that the number of neurons be set to any particular number,and such number may be determined dependent on the complexity of thecurve to be fitted. The formula for such an embodiment may be asfollows, with ‘n’ representing the number of elements:

$\begin{matrix}{Y_{n} = \frac{1}{1 + {\mathbb{e}}^{- {\alpha_{n}{({{\omega_{1n}X} + {\omega_{0n}C}})}}}}} & (8)\end{matrix}$Y _(OUT)=ω₃₁ Y ₁+ω₃₂ Y ₂+ . . . +ω_(3n) Y _(n) +bω _(3b)  (9)

FIG. 18 provides potential results associated with an uncompensatedcrystal oscillator over a temperature range. An uncompensated crystalmay have a total deviation of over 30 ppm. In view of this uncompensatedcrystal performance, it may be helpful to analyze performance of someembodiments of ANN temperature-compensated crystal oscillators. Theanalysis below of possible ANN temperature compensation performanceincludes two different embodiments. The analysis relates to the use of aTCXO, such as a 10 MHz T70 series TCXO's manufactured by GreenrayIndustries, Mechanicsburg, Pa. Such a TCXO may be useful for variousreasons. First, the relatively small size of the TCXO, which may beapproximately 7 mm long by 5 mm in width, may be ideal for certainapplications. Another reason is that the oscillator die may be in aceramic package with a Statek AT-cut strip crystal, which is also in aceramic package, mounted on top. This construction may provide goodthermal design, which has excellent thermal coupling between thetemperature sensor on the die and the crystal.

In a first embodiment, a TCXO's internal compensation may be disabled sothat the crystal is uncompensated and the ANN provides substantially allof the compensation. This may be helpful in evaluating the ANN as aprimary compensation schema for AT-cut strip crystals. The temperaturerange for the analysis may include approximately −42° C. to +86° C. TheANN may then be trained using this data set, and the oscillator run overa temperature range again with the ANN compensation engaged, to evaluatethe ANN compensation performance.

In a second embodiment, compensation may be done using a similar TCXOand crystal type as described for the first embodiments, but with theTCXO's internal compensation engaged. For example, the TCXO may becompensated with an internal 5^(th)-order polynomial generator. Suchanalysis may be helpful in evaluating the ANN as a secondarycompensation schema for state of the art TCXO's. As in the firstembodiment, the temperature range may include approximately −42° C. to+86° C. The data from such a run can then be used to train the ANN. Theunit may then be rerun over the temperature profile with the ANN engagedto evaluate the ANN compensation performance.

In FIG. 18, it is shown that in some embodiments an uncompensatedoscillator may have a deviation over temperature of approximately+/−15.74 ppm. Using this data to train the ANN in the first testembodiment, including, e.g., an ANN with 25 neurons, FIG. 19 showspotential results after training the network and applying thecompensation. The deviation after the ANN compensation is applied may beapproximately +/−0.035 ppm. Such compensation may represent animprovement of approximately 450 to 1 over certain conventional methods.

One consideration in determining what type of oscillator to use forprimary temperature compensation is how much of the oscillator'spullability is used by the compensation. In some embodiments,compensation uses almost all of the oscillator's pullability, leavingthe user little or no ability to adjust the frequency. The residualerror shown in FIG. 18 still has significant content that could be fitif more neurons were used in the network. However, due to the limit onpullability, better compensation may not be realizable on such anoscillator. In some embodiments, a different oscillator design may beemployed for the purpose of primary temperature compensation. Forexample, it may be desirable for an oscillator utilized to be designedto have enough frequency pull to accommodate temperature compensation aswell as user frequency adjust.

FIG. 20 shows characterization data of the compensated oscillator priorto the ANN compensation being applied. In FIG. 20, the uncompensatedoscillator may have a deviation over temperature of approximately+/−0.102 ppm. The data in FIG. 20 may correspond to the residual errorfrom a 5^(th)-order polynomial compensation scheme. This data, orsimilar data, may be used to train the ANN. FIG. 21 shows potentialresults after training the network and applying the compensation.Deviation after the ANN compensation is applied may be approximately+/−0.005 ppm. Such compensation may represent an improvement inperformance of approximately 20 to 1, and may be accomplished using anANN with approximately 33 neurons.

These results may be considered encouraging in several ways. Forexample, as a primary compensation medium the ANN may provide astability better than what is achievable with current state of the artTCXO's, such as about +/−0.1 ppm over the −42° C. to +86° C. temperaturerange. As a secondary compensation medium, with a starting condition ofa compensated TCXO, an ANN may provide improved performance than can beachieved conventionally. Such a system may provide improvement as muchas a 20 to 1, or more, which may rival OCXO's which are generally largerand have much larger power consumption. For comparison purposes, FIG. 22shows a traditional 5′^(h)-order TCXO compensation and the ANN secondarycompensation on the same plot and scale to provide a sense of potentialimprovement.

While the test results shown above are impressive with regard to an ANNcompensation system's ability to reduce deviation in an oscillator, asystem including temperature compensation may still experience variousundesirable effects. For example, trim effect is a skewing of thefrequency versus temperature performance as the control voltage (EFC) isadjusted by the user. As an electronic oscillator ages, its resonantfrequency often changes. Adjusting the EFC may allow a user to restorethe frequency output of the unit, but such adjustment may have adeleterious effect on temperature performance. This effect is oftentolerated because there are currently few practiced solutions for thisphenomenon.

FIG. 23 provides a block diagram of an ANN system that compensates fortrim effect as well as temperature, and FIG. 24 shows a schematicdiagram of such an ANN system. In the embodiment of FIG. 23, atemperature sensor may reside inside the unit as near the resonator aspossible to reduce temperature gradients between the sensor and theresonator. Inputs to the ANN may include the temperature sensor as wellas a user-controlled EFC. The ANN may learn the oscillator's temperatureand trim effect characteristics through a training process and allow acorrection voltage to be created that is then fed into the controlvoltage input (EFC) pin of the oscillator. The correction voltage maymove the frequency to the desired offset of the user and may compensatefor undesirable trim effect.

In FIG. 24, ‘X’ represents the temperature sensor input, ‘V’ representsthe EFC input, and b and C represent some constants which can be anyreal number. The respective weights may be actively solved for. In someembodiments, for ease of calculation, they may be set to 1. It is notnecessary that the number of neurons be set to any particular number,and such number may be determined dependent on the complexity of thecurve to be fitted. The formula for such an embodiment may be asfollows, with ‘n’ representing the number of elements:

$\begin{matrix}{Y_{n} = \frac{1}{1 + {\mathbb{e}}^{- {\alpha_{n}{({{\omega_{1n}X} + {\omega_{2n}V} + {\omega_{0n}C}})}}}}} & (10)\end{matrix}$Y _(out)=ω₃₁ Y ₁+ω₃₂ Y ₂+ . . . +ω_(3n) Y _(n) +bω _(3b)  (11)

FIGS. 25 and 26 provide test results associated with a systemconstructed in accordance with FIGS. 23 and 24. FIG. 25 shows an ANNtemperature-compensated crystal including deviation resulting trimeffect. The trim effect has a total deviation of about 1.6 ppm in theresults shown in FIG. 25.

FIG. 26 shows the resulting deviation after the ANN compensation circuithas been trained. In the system tested, the trim effect was compensatedto a total deviation of about 0.25 ppm.

FIG. 27 depicts a block diagram representing a system that is configuredto compensate for frequency drift during warm-up after power-up, as wellas long-term aging, as the oscillator is powered up for a period oftime. In some embodiments, the system includes three inputs. The thirdinput may be elapsed time which may be needed to fit time-relatedfrequency drift. An exemplary schematic for a three-input system isshown in FIG. 28.

FIG. 29 illustrates an embodiment of a four-input network forcompensating acceleration and vibration sensitivity. While the system ofFIG. 29 compensates for both acceleration and vibration, systems may beconfigured to compensate for such parameters individually. In someembodiments, the fourth input is an accelerometer reading, such as amulti-axis accelerometer, which senses and inputs the accelerationand/or vibration the oscillator is feeling. The system may includeinputs from any number of accelerometers.

FIG. 30 illustrates a block diagram of an embodiment of a four-inputnetwork that may compensate for pressure effects on an oscillator.Frequency change as a function of pressure may occur in applications ataltitudes where air pressure changes induce a frequency shift. Such afrequency shift may be due to field changes from proximity effects ofthe housing deforming under differing pressures.

FIG. 31 illustrates a block diagram of a four-input system that isconfigured to compensate for hysteresis effects. Hysteresis effects maybe distortions of the frequency-versus-temperature curve that depend onthe direction of change of temperature. For example, thefrequency-versus-temperature curves may be different for an oscillatorrunning hot to cold versus one running cold to hot. There also may be atime element to this effect—e.g., the slower the change, the lesspronounced the effect may be; as a result, a change in temperature pertime input may be needed to accurately describe the solution space.

In addition to the embodiments disclosed above, ANN architecture may beused as an oven controller. Such use may be suitable for oven-controlledcrystal oscillator (“OCXO”) and double oven OCXO (“DOCXO”) applications.Block diagrams of these embodiments are shown in FIGS. 32 and 33discussed below. Oven-controller applications may be implementedindependently of, or in combination with, compensation networks, such asthose described above.

FIG. 32 illustrates a block diagram of an embodiment of a systemincluding an ANN oven-controller, as well as ANN temperature and trimeffect compensation. FIG. 33 illustrates a block diagram of adouble-oven controller. In some embodiments, an oven-controller systemdesigned in accordance with embodiments disclosed herein providesdouble-oven performance with a single oven.

The systems tested, as described above, may be implemented usingidealized software models. For example, an input may be converted froman analog voltage to a digital word, and the output converted back to ananalog voltage. Furthermore, embodiments disclosed herein may berealized using hardware models as well. In some embodiments of hardwaremodels, real world transistors and DAC POTS are used, which may allowfor evaluation of resulting degradation of hardware components. In someembodiments, systems are implemented using an integrated model embodiedin application-specific integrated circuit (ASIC).

Trim Effect

As discussed above, trim effect is a skewing of the frequency versustemperature performance of a crystal oscillator as the frequency ispulled (trimmed) away from the oscillator's nominal frequency. Thiseffect is often ignored due to an inability to reduce or eliminate theskewing. Trim effect manifests because the compensation voltage for theTemperature Compensated Crystal Oscillators (TCXO's) is derived with thecontrol voltage (or EFC) at a fixed value (generally ½ of the supplyvoltage) and applied to a varactor. The varactor's reactance varies withthe voltage applied, which results in a change in load capacitance(C_(L)). The frequency versus C_(L) relationship is nonlinear.

This nonlinear relationship leads to degradation in frequency versustemperature performance when the TCXO's control voltage is changed to avalue that is different than what it was compensated with. In practicaluse, TCXO's are commonly adjusted for aging drift. The recalibration ofthe TCXO for aging will often inadvertently cause the TCXO's frequencyversus temperature performance to be degraded.

The present disclosure describes example trim effect compensationschemes using an Artificial Neural Network (ANN). For example, themethod developed by Greenray Industries has successfully compensatedStatek AT-cut strip crystals to stabilities of better than +/−15 ppbover the industrial temperature range of −40 to +85° C. (including a+/−5 ppm trim range).

ANN Overview

A neural network can be described in general as a machine designed tomodel how the brain performs a certain task or function. This “machine”may achieve this modeling through the interconnections of many simpleprocessing units or neurons. In nature, the neurons are biological nervecells which have electrical and chemical responses to given stimuli. Inan artificial neural network, the neuron may be modeled in hardwareand/or software.

As discussed above with reference to FIG. 3A, in an example embodiment,the non-linear model of a neuron may be represented by Equations (1) and(2) reproduced below.Z=(Σ_(i=1) ^(n)ω_(i) X _(i))+ω_(b) b  (1)Y=Φ(Z)  (2)

The inputs X₁ through X_(n) are multiplied by the individual synapticweights ω₁ through ω_(n). There is a bias b, which can have its ownweight ω_(b). These products are summed together, before being subjectedto the activation function. The activation function (e.g., Φ) can benearly any function the designer sees fit to use, and is often chosen tosuit the application. The purpose of the activation function is to limitthe amplitude of the output to some finite value.

For simplicity, the example shown in FIG. 3A may be condensed such thatthe summation and the activation function are both contained in a singlebubble.

Activation Function

As mentioned above the activation function Φ can be any function thedesigner desires. In one embodiment, the activation function selectedfor the task of oscillator temperature compensation is the unipolarsigmoid function, shown in (12). In Equation (12), α is a slope controlvariable, and z is defined in Equation (1).

$\begin{matrix}{\Phi = \frac{1}{1 + {\mathbb{e}}^{- {\alpha{(z)}}}}} & (12)\end{matrix}$

For a two input network, the sigmoid function becomes a threedimensional sigmoid sheet as seen in FIG. 34. The variables affect thethree dimensional sigmoid in the same way as its two dimensionalcounterpart except that there are now two inputs each with independentweights. These two independent weights can rotate or twist the sigmoidsheet in three dimensional space.

In some trim effect compensation schemes, the ANN may have two inputs,x₁ and x₂, where the input x₁ is a voltage that corresponds to thetemperature of the oscillator and the input x₂ is the user appliedcontrol voltage. This two input network is shown in FIG. 35.

The network structure in FIG. 35 can be described by Equation (13).

$\begin{matrix}{Y_{out} = {\omega_{out}\left\lbrack {\left( {\sum\limits_{i = 1}^{n}{\omega_{i}y_{i}}} \right) + {\omega_{bout}b_{out}}} \right\rbrack}} & (13)\end{matrix}$

Each neuron (y₁ through y_(n)) in the input layer has a sigmoidactivation function as shown in Equation (12). The input x₁ is a voltagerepresenting temperature, x₂ is the user applied control voltage, andthe output will be a representation of the deviation in ppm.

Network Training/Learning

The weights of the ANN structure are what ultimately determines itsresponse and therefore must be determined in some fashion. The processof “solving” for these weights may be referred to as learning ortraining. The learning or training refers to a process in which anenvironment that includes a neural network stimulates the neural networkto adapt the parameters of the neural network. There are many differentways to “train” neural networks, and the method selected may beapplication dependent. For example, in some embodiments described in thepresent disclosure, a method of least squares may be implemented using agradient reduction search.

Trim Effect Compensation

In an example embodiment, trim effect compensation using an ANN isimplemented using two neural networks. The first network is responsiblefor compensating the oscillator at center control voltage. The secondnetwork is responsible for the trim effect compensation. The trim effectcompensation network may be placed in parallel to the temperaturecompensation network, and the two compensation voltages may be summedtogether. FIG. 36 shows the block diagram of this example dual networkconfiguration. In some embodiments, both the center temperaturecompensation and the trim effect compensation can be accomplished by thesame ANN. However, the complexity of the network to solve the higherdeviation three dimensional trim effect solution space makes a singleANN design impractical for many applications. The gradient reductionsolver can easily become “stuck” in a local minima, degrading the levelof fit for the solution. Better results have been achieved by firstcompensating the oscillator at center control voltage, and then applyinga secondary trim effect compensation ANN. Thus, many implementations mayhave one ANN for center temperature compensation and another ANN fortrim effect compensation.

An example of hardware that may be used to accomplish the temperatureand trim compensations is shown in FIG. 37. The trained weights for theANN may be stored in the EEPROM 3702. Each neuron may have 4 variablesthat are being trained (e.g., ω₁, ω₂, b, and α). However, in the exampleof Equation (12) of the sigmoid, α is distributed in the quantity z.After training, this distribution can be done so that what is stored inthe EEPROM are the quantities αω₁, αω₂, and αb. This reduces the storagerequirements for each neuron down to three quantities. The network as awhole may also store a ω_(bout) and a ω_(out) value.

In the example of FIG. 37, the microprocessor 3704 samples thetemperature sensor using the ADC 3706. The ADCs 3706 and 3708 may thenapply inputs to the microprocessor 3704 which calculates the ANN'sresponse, and the output can be converted back into a voltage using theDAC 3710. With the current hardware, seven corrections per second can beapplied to the oscillator 3712 providing excellent tracking under fasttemperature ramp conditions. The low pass filter 3714 may be provided toprevent the output frequency from having a step function response to thecorrections.

Prototype Testing

Two different compensations were performed in the initial evaluation ofthe technology. The first compensation was to use one of GreenrayIndustries 10 MHz ANN100-1 oscillator. This oscillator was designedaround the ANN hardware in FIG. 37. The ANN100 series uses a GreenrayIndustries T70 TCXO as the oscillator. The T70 was chosen because theoscillator die is in a ceramic package with a Statek AT strip crystal.which is also in a ceramic package, mounted on top. This constructionprovides a very good thermal design, which has excellent thermalcoupling between the temperature sensor on the die and the crystal. Alsothe Statek AT Cut crystals have proven to be very good for lowperturbations and low thermal hysteresis.

The first compensation done in evaluation of the hardware was to use theTCXO's internal compensation followed by a secondary compensation usingthe ANN methodology described above (with the control voltage set to ½V_(cc)). This was done to provide a low deviation frequency versustemperature performance at center control voltage, which makes thesolution space for the trim effect compensation easier to fit. Thesecond compensation employed was another ANN to provide the trim effectcompensation as described in connection with FIG. 36 above.

ANN Compensation Results

FIG. 38 shows the results of the center control voltage frequency versustemperature compensation and the resulting trim effect. In FIG. 38, thecenter control voltage of +2.5V provides a very flat response overtemperature and as the voltage is trimmed the deviation becomesincreasingly worse.

FIG. 39 shows the results after implementing the trim effectcompensation ANN. The deviation after the ANN trim effect compensationis applied is +/−1.0.7 ppb including trim effect. This is a compensationof 17 to 1. To achieve this compensation, in one embodiment, 10 neuronswere used.

By implementing these two compensations, a TCXO has been created thathas a frequency versus temperature performance of +/−10.7 ppb over theindustrial temperature range (−40° C. to +85° C.). This is inclusive ofthe trim effect generated by trimming the oscillator +/−5 ppm. Thisperformance rivals that of many OCXO's, however the power consumption ofthe ANN100 is less than 100 mW.

The addition of the trim effect compensation to the ANNCXO improves theperformance of the oscillator beyond what is achievable with othertraditionally employed compensation methods, demonstrating thetechnology's robustness as a curve fitter.

The approach shown above outlines an example means for using an ANN toprovide temperature compensation and trim effect compensation for acrystal oscillator. From the above data, it can be seen that the ANNprovides a medium with which to fit very complex multidimensional data.This allows for an embeddable compensation schema to be employed thatcan improve the frequency versus temperature performance beyond what isachievable with traditional methods. It also allows for the compensationof trim effect which is generally present in TCXO's (e.g., that have acontrol voltage).

ANN compensation provides greater than an order of magnitude improvementover the 5^(th)-order polynomial compensation. This realized performanceimprovement over the polynomial style of compensation allows a very lowpower consumption and relatively small device to have temperaturestabilities that rival many existing OCXO's. The phase noise performanceachieved using this approach may be the same as that of a TCXO. In someimplementations, the advantage provided in footprint and powerconsumption (<100 mW) will allow the ANNCXO to replace OCXO's inapplications that are primarily concerned with temperature performanceand may be less concerned with phase noise.

Building on the ANN temperature compensation, the trim effectcompensation demonstrates how more complex multidimensionalenvironmental effects can be compensated. This insight allows theapplication of ANNs to compensate for thermal hysteresis, warm-up, andpossibly aging. The superior curve fitting nature of the ANN effectivelyallows the designer to fit and compensate for any frequency deviationthat can be sensed. As the solution space becomes more complex and spansmore dimensions, the number of neurons needed to effectively fit thatspace will grow drastically. Thus, in some embodiments, the amount ofmemory for storing weights may also grow.

Control Voltage Linearity Compensation

Voltage-Controlled Oscillators and Voltage-Controlled CrystalOscillators (“VCXO”) offer wide pull ranges, often times in excess of100 ppm of adjust. This wide pull allows VCXOs to be used in widecapture range phase-locked loop (“PLL”) applications. A draw back tocreating a VCXO with a large amount of pull is that oftentimes the pullbecomes nonlinear. This nonlinearity can vary from several percent toupwards of 20% depending on the VCXO design. VCXO control voltagenonlinearity can cause problems with the PLL maintaining lock over theentire capture range. There is a need for a method of compensating aVCXO's control voltage to create a linear response. By using anartificial neural network, the VCXO control voltage can be shaped as theuser sees fit providing custom pull ranges and extremely linear controlvoltage transfer functions. Through the use of an artificial neuralnetwork, VCXO control voltage tuning linearity can be achieved withbetter than 0.5% linearity over the entire tuning range of +/−75 ppm.This is more than a 10 fold improvement over the inherent linearityperformance of the VCXO design examined, as shown for example in FIGS.51A and 51B. FIG. 51A is a graph illustrating the deviation in anelectronic oscillator before implementing control voltage linearitycompensation. FIG. 51B is a graph illustrating the deviation in anelectronic oscillator after implementing control voltage linearitycompensation.

Some embodiments of the ANN compensation described above are used to fitan oscillator's frequency response to given environmental and electricalstimuli and create a correction (or compensation) voltage for thefrequency change. In those embodiments, a gain block representing theoscillators control voltage sensitivity is used to convert the output ofthe neural networks into a correction voltage; as shown for example inFIGS. 15, 23, 27, 29, 30, 31, 32, 33, and 36. The output of the neuralnetwork is in parts per million (“ppm”), the gain block has units of−mV/ppm, and the units of the resulting compensation voltage is −mV.

The embodiments disclosed above provide beneficial results, and can befurther improved by accounting for the fact that an oscillator's controlvoltage sensitivity is not constant. The control voltage sensitivitychanges with control voltage level (and to a lesser extent withtemperature), and creates some error in the compensation provided by theembodiments described above. This error can be dealt with by submittingthese remaining residual errors to another ANN to fit and effectivelycompensate out this assumption error. The issues with this approach areprocess time and uncertainty. By having to refit and retest, the processtime is at a minimum doubled which increases costs and reducesmanufacturability. The uncertainty can be caused by not being able toaccurately predict the outcome of any particular test because thecontrol voltage sensitivity is not constant. The embodiments describedbelow combat both of these aspects increasing manufacturability andreducing uncertainty.

Embodiments relating to the linearity compensation ANN compensate for anoscillator's inherent change in control voltage sensitivity (−mV/ppm).This can be done using the same hardware shown, for example, in FIG. 37for the ANN temperature and trim effect compensation.

In the hardware schema shown in FIG. 37, the user's control voltage orEFC is sampled with an analog-to-digital converter or ADC and summedwith the compensation voltage for temperature and trim effect, thenconverted back to a voltage by the digital-to-analog converter or DAC.For the purposes of linearity compensation, the user's control voltagecan be sampled with the ADC and mapped via the neural network to providea DAC output which produces a very linear (nearly constant slope)control voltage response.

FIGS. 40 and 41 show a 125 MHz VCXO EFC pulling curve, and linearitycalculations. As shown in FIGS. 40 and 41, the VCXO has more than 150ppm of pullability over the whole DAC range which is equivalent to 0 to3.3 V on the user EFC pin, and is also extremely non-linear. Thelinearity over the whole 0 to 3.3V pull range is 21.91% (as per MIL55310linearity formula). Even if the area of interest was limited to −50 to+00 ppm, the linearity is still 8.35% (as per MIL55310 linearityformula).

It is desirable to have the frequency pulling over the entire DAC rangeto be as linear as possible while giving the user as much pull aspossible. The ANN/compensation schema block diagram shown in FIG. 42 isconfigured to achieve this.

From a methodology standpoint, a target pullabilty is first established.In one embodiment, a desired pull range can be −50 to +100 ppm from 0 to3.3 volts. In other embodiments, the pull range could be other rangesthat are different, broader and/or narrower than this one. For example,the ranges could include a pull range of −100 to +200 ppm from 0 to 5 or6 volts. In other embodiments, other ranges can be used includingwithout limitation ±50 ppm from 0 to 3.3 volts, ±50 ppm from 0 to 5volts, ±100 from 0 to 3.3 volts, and ±100 from 0 to 5 volts.

After a target pullability is established, the user EFC Voltage is sweptover the entire range and sampled with the ADC at predefined steps; atthese defined steps, in one embodiment, the resultant frequency is alsosampled. The deviation of the resultant frequency from the target iscalculated and a hunt algorithm can be used to determine the DAC valueneeded to set the oscillator to the desired frequency. This creates anADC value (user EFC) and DAC value (desired frequency) map. For theembodiment with a target pull range of −50 to +100 ppm from 0 to 3.3volts, the ADC and DAC values are shown in FIGS. 43 and 44. FIG. 43illustrates the ADC and DAC values versus the desired output frequency.FIG. 44 illustrates the ANN input versus output relationship with theinput being the user sampled EFC and the output being the DAC value ittakes to achieve the desired output frequency in one embodiment.

An ANN using sigmoid activation functions and linear summer on theoutput stages, as described in more detail in the disclosure above, isused to fit the data (e.g., of FIG. 44). After the ANN has been trainedwith the data (e.g., of FIG. 44), the values are stored in theelectrically erasable programmable read-only memory or EEPROM shown inthe hardware scheme of FIG. 37.

Another sweep of the user EFC is done to validate how well the linearitycompensation performs. In one embodiment, the user EFC voltage may beswept again over the entire target pull range and sampled at the samepre-defined steps as the initial sweep used to create the ADC value/DACvalue map. As the user EFC voltage is being swept, the ANN for linearitycompensation can calculate a neural network output signal based in parton the user EFC voltage and the DAC values and test data provided to theANN. A correction voltage may be determined based in part on the neuralnetwork output signal and provided to the oscillator. For eachpre-defined step, the frequency output of the oscillator can be sampled.For the embodiment with a target pull range of −50 to +100 ppm from 0 to3.3 volts, the results of such a sweep can be seen in FIG. 45; in thisembodiment, the new compensated linearity is 0.362% from 0 to 3.3 V ofuser adjust. From the perspective of EFC Voltage, the results shown inFIG. 45 is a compensation of 60.5 to 1, and from the perspective ofoutput frequency, this is a compensation of 23.1 to 1. This linearitycompensation makes the EFC sensitivity constant with a nearly constantslope over the whole input range.

In other embodiments, the ANN linearity compensation can be combinedwith the compensations of an ANN Trim Effect Compensation. The blockdiagram for these other embodiments are shown in FIGS. 46 and 47. FIG.46 shows the ANN linearity compensation combined with ANN temperaturecompensation. FIG. 47 shows ANN linearity compensation combined with anynumber of ANN compensations or other compensation voltages.

In other embodiments, the ANN linearity compensation can be applied tooven-controlled oscillators or OCXOs which are at least an order ofmagnitude more stable; the change in gain from temperature plays a roleas well and, as shown for example in FIGS. 48, 49 and 50, a 2 inputnetwork is used to compensate linearity for user EFC input level andtemperature.

Because the input to the linearity compensation is in ADC bits, ANNs forcompensating variables (e.g., temperature compensation and trim effectcompensation, etc.) can be modified to change their output. As describedfurther above in connection with Trim Effect Compensation using ANN, theANNs output is a fit of the frequency, which can be converted tovoltage. In the case of linearity compensation, a DAC value is fitagainst frequency outputs of an electronic oscillator for whateverparameter that needs to be compensate for. This combined effect, whichaccounts for nonlinearity in control voltage function, allows for a moreaccurate prediction of the outcome of a temperature run and also allowsfor a very good compensation in one pass through an ANN as opposed toseveral runs that, in prior embodiments, were needed.

Other Considerations

The concepts disclosed herein have applicability in a broad range ofsystems and are not limited to the systems described above. For example,environmental effects measurable by an electrical sensor may becompensated in a manner similar to those discussed above. In someembodiments, additional features to be compensated require additionalinputs to the network and, perhaps, more neurons because the solutionspace may become more complex with each additional feature.

Many other variations than those described herein may be apparent fromthis disclosure. For example, depending on the embodiment, certain acts,events, or functions of any of the algorithms described herein can beperformed in a different sequence, can be added, merged, or left out alltogether (e.g., not all described acts or events are necessary for thepractice of the algorithms). Moreover, in some embodiments, acts orevents can be performed concurrently, e.g., through multi-threadedprocessing, interrupt processing, or multiple processors or processorcores or on other parallel architectures, rather than sequentially. Inaddition, different tasks or processes can be performed by differentmachines and/or computing systems that can function together.

The various illustrative logical blocks, modules, and aigorithm stepsdescribed in connection with the embodiments disclosed herein can beimplemented as electronic hardware, computer software, or combinationsof both. To clearly illustrate this interchangeability of hardware andsoftware, various illustrative components, blocks, modules, and stepshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular application and design constraints imposedon the overall system. The described functionality can be implemented invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the disclosure.

The various illustrative logical blocks and modules described inconnection with the embodiments disclosed herein can be implemented orperformed by a machine, such as a general purpose processor, a digitalsignal processor (DSP), an application specific integrated circuit(ASIC), a field programmable gate array (FPGA) or other programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. A general purpose processor can be a microprocessor,but in the alternative, the processor can be a controller,microcontroller, or state machine, combinations of the same, or thelike. A processor can also be implemented as a combination of computingdevices, e.g., a combination of a DSP and a microprocessor, a pluralityof microprocessors, one or more microprocessors in conjunction with aDSP core, or any other such configuration. Although described hereinprimarily with respect to digital technology, a processor may alsoinclude primarily analog components. For example, any of the signalprocessing algorithms described herein may be implemented in analogcircuitry. A computing environment can include any type of computersystem, including, but not limited to, a computer system based on amicroprocessor, a mainframe computer, a digital signal processor, aportable computing device, a personal organizer, a device controller,and a computational engine within an appliance, to name a few.

The steps of a method, process, or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module executed by a processor, or in acombination of the two. A software module can reside in RAM memory,flash memory, ROM memory, EPROM memory, EEPROM memory, registers, harddisk, a removable disk, a CD-ROM, or any other form of non-transitorycomputer-readable storage medium, media, or physical computer storageknown in the art. An exemplary storage medium can be coupled to theprocessor such that the processor can read information from, and writeinformation to, the storage medium. In the alternative, the storagemedium can be integral to the processor. The processor and the storagemedium can reside in an ASIC. The ASIC can reside in a user terminal. Inthe alternative, the processor and the storage medium can reside asdiscrete components in a user terminal.

While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it will beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As will berecognized, certain embodiments of the inventions described herein canbe embodied within a form that does not provide all of the features andbenefits set forth herein, as some features can be used or practicedseparately from others.

What is claimed is:
 1. A system for compensating the effects of controlvoltage on an electronic oscillator to achieve a target frequency outputfrom the electronic oscillator, the system comprising: a neural networkprocessing module operatively coupled to an input signal, the neuralnetwork processing module comprising a plurality of artificial neuronmodules, wherein each of the plurality of artificial neuron modules isconfigured to receive the input signal and one or more pre-calculatedDAC values and provide an output signal, for use in controlling thefrequency output of the electronic oscillator, based at least in part onthe input signal and the one of the one or more pre-calculated DACvalues, wherein the one or more pre-calculated DAC values are derived atleast in part from test data that corresponds to outputs generated bythe electronic oscillator while experiencing a range of control voltagevalues.
 2. The system of claim 1, wherein the input signal is related toa control voltage that corresponds to a value from the range of controlvoltage values.
 3. The system of claim 2, further comprising a summationmodule configured to generate the input signal based on a summation ofat least the control voltage and an output signal from a second neuralnetwork processing module, the second neural network processing modulecomprising a plurality of artificial neuron modules, wherein each of theplurality of neuron modules of the second neural network processingmodule receives one of one or more pre-calculated weights.
 4. The systemof claim 3, wherein the output signal from the second neural networkprocessing module is a correction signal based at least in part on theone or more pre-calculated weights and an output signal from theelectronic oscillator.
 5. The system of claim 4, wherein the outputsignal from the electronic oscillator corresponds to a temperature of atleast a portion of the electronic oscillator.
 6. The system of claim 4,wherein the output signal from the electronic oscillator corresponds toan age of the electronic oscillator.
 7. The system of claim 2, whereinthe neural network processing module is further configured to receive asecond input signal, the second input signal relating to a temperatureof at least a portion of the electronic oscillator.
 8. The system ofclaim 1, wherein the input signal corresponds to a temperature of atleast a portion of the electronic oscillator.
 9. The system of claim 1,further comprising a voltage conversion module, wherein the voltageconversion module is configured to convert the output signal from theneural network processing module into a correction voltage, and providethe correction voltage to the electronic oscillator.
 10. The system ofclaim 1, further comprising a memory module, wherein the memory moduleis configured to store the one or more pre-calculated DAC values.
 11. Asystem for providing a correction voltage to at least partiallycompensate for effects of parameters of an electronic device, the systemcomprising: a first neural network processing module comprising one ormore processors and configured to receive a first input signalcorresponding to a parameter of an electronic oscillator; and a secondneural network processing module comprising one or more processors andconfigured to receive the first input signal and a second input signalcorresponding in part to a control voltage and an output signal from thefirst neural network processing module; wherein each of the first andsecond neural network processing modules comprises a plurality ofartificial neuron modules, wherein each of the plurality of artificialneuron modules receives one of one or more pre-calculated weights,wherein the first neural network processing module is configured toprovide the output signal based at least in part on the first inputsignal and the one or more pre-calculated weights, and wherein thesecond neural network processing module is configured to provide acompensated control voltage based at least in part on the first andsecond input signals and the one or more pre-calculated DAC values. 12.The system of claim 11, wherein the one or more pre-calculated DACvalues are derived at least in part from test data that corresponds tooutputs generated by the electronic oscillator while experiencing arange of control voltage values, and wherein the control voltage of thesecond input signal corresponds to a value from the range of controlvoltage values.
 13. The system of claim 11, wherein the one or morepre-calculated weights are derived at least in part from test data thatcorresponds to outputs generated by the electronic oscillator whileexperiencing a range of values of the parameter.
 14. The system of claim11, further comprising one or more voltage conversion modules configuredto convert the output signal from the first neural network processingmodule into a correction signal.
 15. The system of claim 14, furthercomprising a summation module configured to generate the second inputsignal based on the control voltage and the correction signal, and toprovide the second input signal to the second neural network processingmodule.
 16. The system of claim 11, wherein the first input signalcorresponds to a temperature of at least a portion of the electronicoscillator.
 17. The system of claim 11, further comprising a memorymodule, an analog-to-digital converter, a digital-to-analog converter,and a low-pass filter.
 18. The system of claim 17, wherein the memorymodule is configured to store one or more values calculated bymultiplying each of the one or more pre-calculated weights by a slopecontrol variable, and store one or more values calculated by multiplyingone or more bias inputs provided to the first neural network processingmodules by the slope control variable.
 19. A method of compensating foreffects of control voltage on an electronic oscillator to achieve atarget frequency output from the electronic oscillator, the methodcomprising: receiving test data from an electronic oscillator, theelectronic oscillator being subject to a test control voltage, whereinthe test data corresponds to outputs generated by the electronicoscillator while being subject to the test control voltage over a rangeof test control voltages; calculating a plurality of DAC values at leastin part on the test data; providing a current control voltage to aneural network processing module including one or more processors;providing the plurality of DAC values to the neural network processingmodule; using the neural network processing module to calculate a neuralnetwork output signal based at least in part on the current controlvoltage and the plurality of DAC values; determining a correctionvoltage based at least in part on the neural network output signal; andproviding the correction voltage to the electronic oscillator to atleast partially control the frequency of the electronic oscillator. 20.The method of claim 19, wherein said calculating the first plurality ofDAC values is performed prior to said receiving the current controlvoltage.